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ABSTRACT 

Our paper describes the design and delivery of an online interdisciplinary social science research 
methods course (ISRM) for graduate students in sociology, education, social work, and public 
administration. Collaborative activities and learning tookplace in two types of computer-mediated learning 
environments: a closed Blackboard course management system and a public facing “openly-networked 
connected learning” environment designed to facilitate cross-discipline connections, student engagement, 
and digital fluency. A course formative assessment based on student feedback and instructors’ reflections 
informed the lessons learned about the design and delivery of the course. Our assessment suggests that 
many of the connected learning goals can be met through the closed course management system rather 
than through the open platform. 
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Several challenges confront the 21st century 
social science university professor, not the least 
of which are (a) equipping graduate students with 
sufficient research capacity to understand, generate, 
and use scientific knowledge in a world driven by 
multiparadigmatic perspectives, and (b) assessing 
a plethora of technological advancements for use 
in varied learning environments. To address these 
challenges, four social science research methods 
instructors, an instructional designer, and a research 
librarian formed a faculty learning community 
(FLC) to collaborate on the design and delivery of 
an online interdisciplinary social science research 


methods course (ISRM) for graduate students 
in sociology, education, social work, and public 
administration. The ISRM course, characterized as 
an openly-networked connected learning approach, 
leveraged digital tools and technologies to connect 
student learning environments across disciplines, 
home, and community settings (Connected 
Learning Alliance, nd). The course web site was 
built on a public-facing open web site platform 
designed to facilitate student blogging and peer 
learning across disciplines, seamless integration 
of abundant multimedia content (podcasts, videos, 
text, data), and sharing course content with 
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interested university and community members 
(Croteau, 2015). An openly-networked connected 
learning approach supports active collaborative 
learning through student interest-driven projects, 
opportunities for student reflection, and frequent 
feedback from peers and instructors—themes 
identified by Nind, Kilburn, & Luff (2015) in a 
recent analysis of articles about teaching research 
methods. Connected learning is also expected to 
facilitate greater student autonomy in a self-guided 
learning environment (Dron, 2007; McLoughlin 
& Lee, 2010). For instance, a social bookmarking 
tool like Diigo enables students to curate, share, 
and integrate their own learning content into 
the course (Stoeckel & Sinkinson, (2013). Four 
components characterized the new ISRM course 
and distinguished it from the discipline specific 
research methods courses taught previously: 

1) a public-facing course site based on connected 
learning principles; 

2) a structured blogging component; 

3) an interdisciplinary approach; and 

4) an information literacy component anchored 
in library science. 

The purpose of this article is to share the 
lessons learned from a course assessment of this 
learning initiative to contribute to the emerging 
pedagogical knowledge base for teaching graduate 
research methods course content in computer- 
mediated learning environments. First, we identify 
and contextualize some of the critical issues around 
teaching research methods and note how our 
openly-networked connected learning environment 
was informed by and could address these issues: the 
need for increased research capacity in graduate 
students, trends in the technology and pedagogy 
of teaching and learning research methods, the 
benefits of interdisciplinary course delivery, and 
the changing information landscape. Second, we 
describe the design and delivery of the ISRM 
course, highlighting the four new components that 
were incorporated into the course. Next, we present 
the course assessment, including student feedback 
and instructor reflections, that was undertaken 
to better understand the student response to and 
the learning associated with the new course 
components. This assessment informed our lessons 
learned which we organize and present within the 
framework of the new course design components 
offered in the ISRM course. We conclude that the 


glass is half full in that the benefits of two of the new 
components, the interdisciplinary approach and 
the information literacy component, were clearly 
apparent while the two other components, openly- 
networked technology and the associated blogging 
activities, contributed less than we expected. 

THE CHANGING CONTEXT OFTEACHING SOCIAL 
RESEARCH METHODS 

Although the creation and dissemination of 
new knowledge has been the hallmark of graduate 
education for several decades, increasing demands 
for evidence-based practice and research-informed 
policy has generated a new urgency for research 
capacity—defined as the understanding and use 
of science to confront complex social problems in 
today’s rapidly changing world (Nind, Kilburn, & 
Luff, 2015). Research capacity requires that students 
not only master a range of research methodologies 
but also develop insights and proficiencies to apply 
research knowledge in real world settings. This 
is no small task given what we know about the 
apprehension of social science students entering 
their introductory research methods courses 
(Earley, 2013; Zerden, Powers, & Wretman, 2014). 
Unfortunately, little pedagogical guidance or 
empirical evidence is available to help build student 
research capacity (Kilburn, Nind, & Wiles, 2014). 
The recognized need for a pedagogical culture for 
teaching social science research methods in our 
institution prompted the formation of our FLC. 

The impact of technology on the lives of 
educators, students, and academic support personnel 
on university campuses has been prominent and 
pervasive. Of particular importance is the rise, 
reach, and variability of the virtual classroom that 
ranges from hybrid arrangements, blending in- 
person and online learning environments, to fully 
online courses with synchronous or asynchronous 
participation (Allen & Seaman, 2013; McNeal, 
2015). Commercial learning management systems 
like Blackboard, freely-available open source 
platforms like Moodle and Wordpress, and Web 
2.0 technologies like Twitter, blogs, and wikis, 
contribute to this rapidly changing learning 
environment (Cornelius, 2014; Palmer & Schueths, 
2013; Toven-Lindsey, Rhoads, & Lozano, 2015). 

The many forms of participation afforded by 
these technologies fuel the appeal of constructivism 
and connectivism as the pedagogical grounding 
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for online learning approaches (Educational 
Broadcasting Corporation, 2004; Schulze, 2009; 
Stillar, 2012). Courses are designed to be student¬ 
centric, active, and collaborative, bringing peers 
together to solve problems and process new 
knowledge (Bryant & Bates, 2015; Revere & 
Kovach, 2011). Means, Toyama, Murphy, Bakia, 
& Jones (2010) conducted an extensive analysis of 
online learning using a quantitative meta-analysis 
of 99 empirical studies and a narrative synthesis of 
an additional 77 empirical studies and concluded 
that “students in online conditions performed 
modestly better, on average, than those learning 
the same material through traditional face-to- 
face instruction” (p. xiv). Similarly, a majority of 
academic leaders have reported that they considered 
online education “as good as or better” than that in 
face-to-face classes (Allen & Seaman, 2013). 

These technological advancements and 
pedagogical approaches bring new opportunities 
to teaching research methods, one of the most 
challenging, yet understudied, subjects across 
the social and behavioral science curriculum. 
Investigations about teaching research methods 
in online formats are just beginning to surface in 
the literature and focus on a wide range of topics 
including: 

• the exploration of constructivist learning 
approaches to teaching research methods courses 
in a masters of education course (Schulze, 2009); 

• the application of Chickering & Ehrmann’s 
principles of quality education to an online 
undergraduate research methods course in 
healthcare management (Johnson, 2014); 

• the assessment of a hybrid graduate 
research methods course in criminal justice and 
criminology program (Pelfrey & Bubolz, 2014); 

• the adaptation of a traditional doctoral- 
level research methods course in education to 
a completely online Blackboard based course 
format (Lim, Dannels, & Watkins, 2008); and 

• the insertion of online or electronic 
components into face-to-face research courses 
in undergraduate psychology courses (Bates, 
Rodriguez & Drysdale, 2007) and nursing 
curriculums (Sabey & Horrocks, 2011). 

The few studies that compared online to 
face-to-face formats in research methods courses 
reported that there is little difference in learning 
outcomes between face-to-face and online formats 


(Girod & Wojcikiewicz, 2009; Ya Ni, 2013). We 
envisioned our FLC as an ideal forum to expand the 
discussion related to the technology and pedagogy 
of computer-mediated environments by exploring 
innovative strategies in delivering fully online 
research methods courses. 

Interdisciplinary competencies are increasingly 
expected as today’s faculty and graduate 
students navigate a world where disciplinary 
boundaries are blurring and interprofessional 
collaboration is valued (Kemp & Nurius, 2015). 
In interdisciplinary courses, students become 
familiar with other professions’ knowledge and 
scope of practice, develop a respect for others’ 
contributions (Supiono & Berry, 2013), recognize 
the strengths and limitations of discipline-based 
research, and integrate interdisciplinary thinking 
into their research repertoires (Gill et ah, 2015; 
Kecskemethy, 2008; Mabry, Olster, Morgan, & 
Abrams, 2008). Because interdisciplinary research 
is often driven by an orientation toward applied 
research activity (Wolley, Sanchez-Barrioluengo, 
Turpin, & Marceau, 2015), it is particularly relevant 
to graduate students in social work, sociology, 
education, and public administration—the core 
group of students targeted in our FLC. 

Although notably absent in the literature 
on teaching social science research methods, 
information competencies are integral to the 
development of students’ research capacity. In 2000, 
in response to an increasingly complex information 
landscape marked by an explosion of scientific 
journals and unvetted publications (Head, Van 
Hoeck, Eschler, & Fullerton, 2013), the Association 
of College and Research Libraries (ACRL) 
developed Information Literacy Competency 
Standards for Higher Education to help ensure that 
students understand how knowledge is organized 
and know how to search for, critically evaluate, 
synthesize, and use reliable information ethically 
and in such a way that others can learn from this 
knowledge. 

Regardless of discipline, information literacy 
provides a foundation for independent research, 
evidence-based practice, and lifelong learning 
(Andretta, 2005). Unfortunately, graduate 
students are often left to their own devices for 
library research (Rempel & Davidson, 2008). In 
addition to time constraints, faculty often lack the 
knowledge and expertise required to address this 
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research component adequately. Because students 
need guidance that goes far beyond the few 
pages typically offered in introductory methods 
texts (Boote and Beile, 2005; Granello, 2001), 
collaboration between research methods faculty 
and librarians is a logical solution and one eagerly 
embraced by our faculty learning community. 

DESIGN AND DELIVERY OFTHEISRM COURSE 

This FLC was situated in one U.S. southeastern 
urban public research university where connected 
learning was being actively promoted and 
supported by the faculty development center at 
the time. The number, nature, and complexity of 
resources required to initiate the ISRM required a 
systems approach that could be delivered only at the 
university level. The center’s instructional designer 
brought together educators from education, 
public administration, social work, and sociology 
programs who were experienced in teaching 
online graduate research methods courses to create 
an openly-networked ISRM course. A research 
librarian was invited to join the FLC to incorporate 
an information literacy component. 

Although the evidence for a common 
pedagogical approach to teaching research 
methods is limited in the literature (Wagner, 
Garner, & Kawulich, 2011), a review of the 
syllabi used by the FLC faculty members in their 
discipline-specific research courses revealed 
similarity among the courses in terms of course 
objectives, course content, assignments and 
quizzes, small group work, and a basic adherence 
to the cognitive dimension of Bloom’s Taxonomy 
of Learning (Betts, 2008). Each of the instructors 
addressed qualitative, quantitative, and mixed 
methods research designs, but all placed emphasis 
on quantitative methods, primarily because of the 
continued dominance of quantitative methods in 
each of the instructors’ disciplines and because of 
the focus on quantitative methods in the research 
methods texts used by the instructors. Basically, 
the instructors had shared experiences in teaching 
research methods in the long-standing tradition of 
the positivist paradigm (Wagner & Okeke, 2009) 
with respectful (and increasing) attention to the 
interpretivist and mixed methods approaches. 
In addition to these overlapping course design 
elements, which were incorporated into the new 
interdisciplinary course, the FLC group recognized 


the value of two strategies developed independently 
by two of the faculty members and incorporated 
these strategies into the course as well: 1) a food 
for thought question series to encourage higher 
order thinking about the material; and, 2) the use of 
proposal development worksheets to guide students 
through each step of the research process. 

All instructors had experience using 
Blackboard, including features such as discussion 
boards, journaling, and wiki pages; all were familiar 
with Babbie’s research methods text The Practice 
of Social Research (Babbie, 2015), which has been 
widely used in the social sciences. The course 
similarities reinforced the expectation that these 
faculty members could work toward a common 
syllabus and course design that would extend their 
current discipline-specific online research methods 
courses into one course open to students enrolled 
in any of the four academic units. 

Informed by the Digital Media and Learning 
Research Hub and the Connected Learning 
Alliance, the new ISRM course was anchored by 
the following principles: interest-powered (content 
and assignments related to student goals), peer- 
supported (student participation in collective 
learning experiences), shared-purpose (working 
together to achieve a common goal), academically- 
oriented (driven by academic success, civic 
engagement, or professional development), 
production-centered (taking advantage of digital 
media for experiential or hands-on learning), 
and openly-networked (a public facing web site 
that connects classroom learning with real world 
settings across space, time, and multiple spheres). 
The first five principles—interest-powered, peer- 
supported, shared-purpose, academically-oriented, 
and production-centered—were familiar to the 
members of the FLC, who easily found these 
principles embedded in their existing discipline- 
specific online courses by requiring students to 
work together on specific assignments or develop a 
group research project based on a combined area of 
interest. In their previous courses, instruction and 
content on developing the research proposal was 
delivered through the Blackboard Management 
System with students interacting primarily through 
Blackboard discussion forums, Collaborate, or 
wiki tools. 

The sixth principle, openly-networked, 
which inspired the term used to describe the 
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interdisciplinary course, was new to the instructors 
and became an integrating component of the 
course through the use of a customized WordPress 
platform, a widely adopted open source publishing 
and content management platform (Jones & Alida- 
Farrington, 2011). Blogging, another feature 
associated with openly-networked environments, 
was added and located in the WordPress site for 
students to exchange ideas with one another. 
Blogging promotes universal access to activities, 
thoughts, and beliefs almost as they are being 
created (Bouwma-Gearhart & Bess, 2012). 
Shulman’s observation on the nature of learning 
is highly relevant to an openly-networked context: 
“learning flourishes when we take what we think 
we know and offer it as community property among 
fellow learners so that it can be tested, examined, 
challenged, and improved before we internalize it” 
(P-12). 

The course moved from a restricted Blackboard 
site available only to students in a particular 
discipline to an open web site that was available 
to students enrolled in programs from several 
disciplines. In addition, most, but not all, course 
interactions and sources of information were open 
to the public through the WordPress site. To protect 
the privacy that students and faculty expect in a 
graded for-credit course (Mentor, 2007), student 
assessments and grading were available on a 
Blackboard site accessible only to students who 
were registered for credit. This also helped faculty 
focus their time on registered students whose 
mastery of this material would be critical to their 
success in subsequent required research courses. In 
summary, the ISRM course was a blend of open and 
closed platforms. The more generic components 
of the course, such as the syllabus, podcasts, and 
blogging activities, were publicly available through 
the WordPress web site. The components related to 
student monitoring, assessment, and grading were 
confined to the Blackboard site. The primary course 
portal was through the public web site that was then 
linked to Blackboard with the technological support 
of the university; students taking the course for 
credit were registered in the Blackboard system. 

Fifty-two students registered for the course 
and 49 completed it, a 94% retention rate that is 
considered excellent for online courses (Bawa, 
2016). Because cross-disciplinary interaction 
among small groups is critical to interdisciplinary 


course design (Kemp & Nurius, 2015), students 
and the instructors were distributed among four 
cross-disciplinary groups (average of 12 students 
per group) to complete group assignments. Each of 
the instructors maintained primary responsibility 
for grading and providing feedback to students in 
their group, but they met frequently throughout 
the semester to ensure consistency for cross- 
disciplinary group grading. 

As one of the goals of the FLC was to help 
faculty and students gain expertise with digital tools 
and openly-networked resources, each instructor 
created his/her own WordPress web site that was 
then linked to the course’s “mother” web site. 
Students published their blogs to the account of the 
instructor responsible for their cross-disciplinary 
group, which was then linked to the “mother” 
web site; all members of the class could access 
each other’s group sites. The instructional design 
member of the FLC served as the administrator/ 
owner of the course web site; the instructors were 
granted editing capability and the students were 
granted author status to facilitate creating personal 
blogs. 

The course was divided into 12 weekly units 
corresponding to the chapters in the course text 
(Babbie, 2015). Course activities and assignments 
consisted of: weekly text readings, podcasts, and 
other material and online educational resources 
accessed via the course web site and the internet; 
one ungraded library assignment at the beginning 
of the course; completing weekly textbook¬ 
generated quizzes; participating in weekly blogging 
discussions to answer food for thought questions; 
completing five group proposal development 
worksheets (i.e., developing research question, 
sampling, design, etc.); the final group research 
proposal; self-assessment of the group proposal 
process; a multiple-choice midterm; and an article 
critique as the final exam. A semester calendar 
outlining course requirements and due dates was 
posted on the course web site. Students earned one 
point for every quiz completed with 80% correct 
answers, 0-2 points for blog entries (0 for no or a 
minimal post, 1 for satisfactory postings comparable 
to B-level work, 2 for outstanding postings 
comparable to A-level work). The worksheets 
were not graded; it was assumed that diligence in 
completing the worksheets and using the instructor 
feedback would be reflected in the proposal grade. 
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The final proposal and the exams were graded 
on traditional 100% scales. All students earned a 
passing grade in the course. 

A library guide facilitated access to social 
science databases and to videos, web content, and 
tutorials focused on building specific information 
competencies. Each student completed a nongraded 
search exercise that helped them prepare for the 
research proposal by: extracting preliminary 
search concepts from a well-stated research 
question, organizing search concepts using a logic 
grid, identifying databases relevant to the research 
question, choosing appropriate search parameters, 
analyzing the results, and revising strategies 
based on the initial search results. The exercise 
also functioned as a needs assessment tool that 
facilitated individual and group feedback provided 
by the librarian. 

The food for thought questions deserve special 
mention because these questions: anchored the 
blogging experience, which distinguished the 
ISRM as a connected learning course; provided an 
interpretivist-qualitative aspect that was important 
to balance out the quantitative component of the 
course; and contributed to the course assessment 
as discussed in the section below. The questions 
were designed to stimulate conversation, connect 
to real-world settings through electronic resources, 
and foster reflections about different aspects of the 
research process. Examples of food for thought 
questions were: 

• What are common errors of human inquiry? 
Find a magazine, newspaper article, editorial, or 
blog that illustrates one or more of these errors. 
Explain. Be sure to include a quote and/or a link 
to the source. 

• What does trust mean to you? Explain your 
concept of trust and then try to operationalize it 
in some way that would allow it to be measured. 
What types of questions might you ask? 

• Review all the blogs to date posted on the 
class course site. What might you do to analyze 
these blogs? What types of things might you 
want to research and draw conclusions about 
regarding the types of posts, the students who 
post, etc.? 

Instructors commented on the blogs within the 
cross-disciplinary groups but posted the blog grade 
in Blackboard grade center. 


COURSE ASSESSMENT: FEEDBACK FROM STUDENTS 
AND INSTRUCTORS 

As with any educational initiative, gathering 
feedback and reflecting on both the experience 
and the outcomes of the course not only inspires 
continual course improvement but, in this case, 
also formed the basis for a series of lessons learned 
to be shared with other online educators to help 
develop the most effective computer-mediated 
course delivery environments. The process of 
gathering information to improve student learning 
is known as formative course assessment and is 
distinguished from summative course assessment, 
which is aimed at evaluating student performance 
and learning outcomes (Angelo & Cross, 1993; Frey 
& Schmitt, 2007; Taras, 2008). Student feedback 
and instructor reflections formed the basis of the 
formative assessment for the ISRM course, which 
was then used to create the subsequent “lessons 
learned.” 

Student Feedback 

Student feedback comprises an essential 
aspect of any formative course assessment and 
can be gathered in a variety of ways. In this 
ISRM course, we used a midsemester check-in 
and feedback solicited directly by the librarian to 
answer the underlying questions of any formative 
assessment, such as “What is working, what needs 
to be improved, how can it be improved?” (Dixson 
& Worrel, 2016, p. 155). We used responses to 
one of the food for thought blogging questions as 
a formative assessment tool embedded within the 
course learning activity and linked directly to the 
unit of instruction (Perie, Marion, & Gong, 2009). 
A fourth source of student feedback was provided 
by student responses on the end-of-semester 
student course evaluations that were generated by 
the university for the ISRM course. Although the 
relationship between student course evaluations 
and student learning outcomes is debatable, student 
course evaluations are recognized as a viable 
and acceptable tool to assess student learning 
experiences, provided they are not used as a sole 
indicator (Carbone, Ross, Phelan, Lindsay, Drew, 
Stoney, & Cottman, 2015; Vasey & Carroll, 2016). 

Midsemester check-in 

The midsemester check-in occurred six weeks 
into the semester. Students were asked via e-mail 
for feedback on course structure, content, and group 
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processes. Students expressed concerns about 
the format of the self-assessment quizzes as they 
did not allow viewing of the correct answers; the 
blogs, which seemed disconnected from the rest of 
the course; and too much course information being 
scattered over too many locations. Subsequently, 
we (a) corrected the quiz format to allow students to 
view correct answers immediately upon completion 
of the quiz; (b) instituted weekly instructor blogs 
to synthesize and guide student learning and to 
correct inaccurate information posted by some 
students on the blog site; and (c) created new 
customized screencasts or podcasts for each of the 
weekly topics to better aid students in navigating 
content. 

Food for Thought Question 

All 49 students responded to a multiple- 
faceted food for thought question posted toward 
the end of the semester: “What did you learn by 
being active in this course? Did working with an 
interdisciplinary group interested in all forms of 
social research enhance your learning? Did you 
do your work the way other people did theirs? Did 
conversation during collaboration strengthen your 
learning? What changed your mind about research? 
What do you still need to learn to do quality 
research?” Guided by the assumption about good 
classroom assessment that “the type of assessment 
most likely to improve teaching and learning is 
that conducted by faculty to answer questions they 
themselves have formulated in response to issues or 
problems in their own teaching” (Angelo & Cross, 
1993, p. 9), this food for thought question sought 
feedback on issues related to the goals of our FLC 
initiative. We categorized the student responses 
similarly: three categories captured information 
about the important aspects of three of the four 
new components introduced into this course by 
the FLC (interdisciplinary nature, blogging, and 
structure and format); a fourth category captured 
student feedback on the collaborative nature of 
the course because of the social constructivist 
approach that was foundational to the course; 
and a fifth category, the extent of learning, was a 
direct assessment of student’ perception of their 
own learning. To avoid instructor bias, a Graduate 
Research Assistant, unfamiliar with the course, 
coded the blog discussions of each student into the 
five categories noted above. Specifically, the GRA 
created a matrix to organize all comments, words, 


or phrases from each student’s blog associated 
with any of the five categories identified by the 
instructors. Once the student names were removed, 
one of the authors of this paper (who was also one 
of the FLC instructors) reviewed the matrix and 
summarized the findings as described below. 

Regarding the interdisciplinary nature of the 
course, although a quarter of the students thought 
that their learning was not enhanced by having 
students from other disciplines in the course, 75% 
of the students thought that their learning was 
enhanced by the interdisciplinary aspect, noting is 
was “interesting,” “stimulating,” and “refreshing.” 
Students respected others’ ideas, learned with 
and from one another, acknowledged the value 
of different perspectives, and felt they gained a 
broader view of research process. For some, this 
meant selecting a research problem they would not 
otherwise consider; for others, it provided exposure 
to new concepts and/or an understanding of how 
the same concepts may be applied in different 
contexts. In addition, students learned about other 
programs in the university. 

In regard to the collaborative nature of the 
course, students reported that: interacting and 
conversing with the members of their group 
“strengthened” or “enhanced” learning, provided 
a strong sense of “emotional support” from other 
students, encouraged conversations with peers that 
were “helpful” and facilitated “understanding of 
the material” and “clarified concepts and methods,” 
and created a collegial atmosphere with “regular 
communication” with group members who were 
considered “teammates.” In sum, working in a 
team to create the research proposal and conversing 
with team members for support and guidance was 
a vital component of the learning process. 

Regarding the blogging component, which 
was implemented to better align the course with a 
connected learning approach, some students noted 
that blogging was “valuable” and provided some 
insights into how other students understood and 
applied the material to their disciplines. However, 
a majority of students commented that the blogging 
seemed to be an unnecessary burden and most 
agreed that the technology was not user friendly. 
Overall, students indicated a preference for the 
more familiar Blackboard discussion forum as a 
venue for interaction and exchange of ideas over 
the public WordPress blog. 
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The structure and format of the course was 
the fourth category from the blogging assessment. 
Many students found the course to be “fast paced” 
and more “challenging” and “intense,” with more 
detail and emphasis on theory in research methods 
than previous research courses they had completed 
during their undergraduate program. Students 
reported that the workload felt “overwhelming,” 
the use of the various technology formats was 
confusing, and the organization of the course was 
difficult to follow. 

In regard to the fifth category, the extent 
of learning, students shared their perceptions 
of how much they learned in the course. Many 
students reported a stronger appreciation for 
research methodology and the research process, a 
better understanding of the relevance of research, 
and increased confidence in their own research 
abilities. Some noted that they learned “more than 
expected.” Several students noted an enhanced 
ability to read journal articles, develop a research 
question, and critically think about research. 
Learning the differences between quantitative 
and qualitative methods, research ethics and 
terminology, and how to write research proposals, 
were additional strengths. Students recognized 
that their comprehension of research methods was 
increased through completing various assignments, 
“deliverables,” and worksheets and that the proposal 
assignment was especially helpful to understanding 
research methods. Interestingly, students also 
described a process of “self-discovery” whereby 
they recognized greater insight into themselves as 
learners and their own personal learning style as a 
result of the ISRM course. 

Course Evaluations 

Course evaluations in this university were 
electronic and anonymous; they included both 
quantitative and qualitative questions, allowing 
for the calculation of percentages for the fixed 
response questions as well as gathering comments 
about the strengths and weaknesses of the course 
from open-ended questions. Eighty-two percent 
(82%) of the 49 students who completed the course 
also completed the course evaluations. Much of 
the anonymous course evaluation information 
was consistent with the midsemester check-in and 
the blog responses where the students were self- 
identified. However, the criticism of the course 
structure was more pronounced in the course 


evaluations, where over 60% of students disagreed 
or strongly disagreed with the statement that the 
“course material was presented in an organized 
and informative manner.” Comments supported 
this rating: 

•“too many blogs that distracted from the 
other work,” 

•“we should get rid of the blog and just use 
Blackboard,” 

•“too many overlapping assignments,” 

•“the course material is too confusing to 
follow multiple instructors,” 

•“so many requirements per week that I 
became lost in what was expected of me instead 
of trying to understand the material,” and 

•“the format of the online material was 
located in WAY too many places.” 

In the anonymous evaluation, students were 
less favorable about the interdisciplinary nature 
of the course than they had been in the public 
blog about this issue. As one student noted, “the 
interdisciplinary approach didn’t seem to have a 
huge impact for me in this class.” 

On the other hand, most of students felt positive 
about the instructors and their learning. Ninety- 
one percent (91%) “agreed” or “strongly agreed” 
that instructors created an environment of trust 
and fairness and that students were treated with 
courtesy and respect. 

Students noted that the instructors were 
knowledgeable, responsive, provided good 
feedback, and produced good videos and 
podcasts. A few mentioned that they thought one 
or two in-person classes would help convey the 
material. Ninety-two percent (92%) reported that 
they “engaged in critical thinking,” “increased 
knowledge of the subject matter,” and “developed 
greater self-awareness” while 69% reported that 
they “developed significant skills.” Some of the 
positive comments repeated what was noted in the 
blog responses: 

• “I learned a lot about research design and 
the components of a research proposal,” 

• “I feel like I have gained a foundation in 
research and actually have a desire to learn 
more,” and 

• “I can be more critical when reading 
research and ‘evidence based’ programs.” 
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Feedback to Librarian 

The librarian gathered student feedback 
through e-mail and research consultations 
throughout the duration of the course. Students 
noted concerns about their lack of library research 
knowledge and were very appreciative of the help 
navigating the course web site, posting blogs, 
locating research based on a specific methodology, 
gathering background information for the proposal, 
finding the most recent prevalence and incidence 
statistics to justify their research question, and 
choosing topic-specific databases. Groups were 
especially grateful for the proposal development 
support; faculty also acknowledged the librarian’s 
contribution. 

INSTRUCTORS’ REFLECTIONS ON THE ISRM COURSE 

Finally, we acknowledge that “college faculty 
assume a great deal about their students’ learning, 
but rarely make these assumptions explicit or check 
them against the students’ impressions” (Angelo 
& Cross, 1993, p. 7). To offset this deficiency and 
provide a fuller complement for our formative 
assessment, faculty members involved in this 
project participated in an end-of-semester informal 
evaluation session to reflect on their experiences 
with the course. Two major themes emerged. 

Benefit of a Shared Experience: We’re in This 
Together 

The instructors were encouraged by the 
collaborationwith their interdisciplinary colleagues. 
They could reciprocally learn from the challenges 
and share strategies for delivering this content. 
This exchange included insights from their own 
personal experiences as educators and from their 
own disciplinary perspective. The milieu reflected 
a sense of respect for participants’ contributions, 
skills, and knowledge across disciplines and a 
sense of camaraderie and enjoyment, which was a 
welcome relief from the isolation often experienced 
by instructors focused solely on their own courses. 
In addition, the guidance offered by the librarian 
provided valuable troubleshooting, support, and 
technical expertise as questions arose during 
the semester. Library resources and tools were 
recognized as aids to student learning as well as 
additional tools for the instructors’ teaching that 
supported the course learning objectives. 


Problems with Platform: Public in Nature and 
Personally Challenging 

The instructors saw the WordPress terminology 
as obscure and the navigation tools were difficult 
to manage, particularly in comparison to the more 
familiar Blackboard. The multiple linked web 
sites, created by the instructional design member 
to increase digital literacy, were especially 
burdensome as the instructors spent too much time 
and effort learning the various WordPress features 
and guiding their students in a similar learning 
process. The consensus among the FLC was that the 
major learning objectives for the course, including 
some of the connected learning goals, could be 
accomplished as easily through Blackboard. 

Related to pedagogy, the public nature of 
blogging presented challenges to faculty. They 
worried that the frustrations and anxieties many 
students experience as they work to master 
research content (Early, 2013) may be particularly 
exposed in this public forum. Faculty members 
were especially sensitive to correcting students 
for effective learning without being overly critical 
in such a public context. In addition, while this 
course was open to public viewing, very few 
visitors actively engaged with the course, making 
the benefit of having a public face in the learning 
environment nearly inconsequential. 

LESSONS LEARNED 

The lessons learned from the student feedback 
and instructor reflection are organized around the 
four components added to the discipline-based 
graduate research methods courses previously 
taught by the instructors: an interdisciplinary 
approach, an openly-networked web site format, a 
blogging component, and an information literacy 
component. Although none of these course design 
components had undergone prior extensive 
evaluation, each had merited attention in the 
literature as viable and valued considerations for 
online or connected learning courses. 

Interdisciplinary Nature of the Course 

Members of the FLC agreed that the 
collaboration provided a rich source of information, 
support, and, most importantly, affirmation about 
teaching research methods across social science 
disciplines. The similarities among the existing 
research methods courses promoted ready adoption 
of most of the prior activities or assignments for 
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the new course design. To this array of assignments 
(quizzes, readings and screencasts, accessing 
OERs, proposal development, and exams) we added 
the blogging component without due regard for 
the cumulative effect that so many different types 
of learning strategies might have on the overall 
student and faculty experience in the course. 

A more in-depth assessment of how the activities 
triangulated with or reinforced each other might 
have minimized the confusion and frustration 
that students expressed about the overlapping 
assignments and excess of requirements. For 
example, either the blogging activity or the 
proposal worksheets might have been adequate to 
develop research capacity. Similarly, perhaps the 
self-assessment quizzes or one exam rather than 
both a midterm and a final might provide ample 
evidence of students’ mastery of course content. 

Although students were clearly dissatisfied with 
the number, type, and organizational layout of the 
course activities produced by our interdisciplinary 
efforts, they overwhelmingly reported that they 
did learn research, some more than they had ever 
expected. Would this perceived level of learning 
have occurred with fewer activities? Were all 
activities necessary to achieve the learning goals? 
In the online environment, there is a real need to 
critically assess each teaching activity within the 
context of the other activities. This is an important 
lesson to learn, especially in online education where 
the proliferation of technological tools coupled 
with the growing popularity of interdisciplinary 
course development, can create temptations to add 
indiscriminately both the tried and true as well 
as the latest bells and whistles. We know of no 
examinations of the impact of the “constellation” 
of teaching activities for online student learning, 
and we believe that this would be a fruitful area of 
exploration in the developing pedagogical culture, 
not only for teaching research methods specifically 
but also for any computer-mediated course. 

Whereas some students devalued the 
interdisciplinary benefits of the course in the 
anonymous course evaluations, most were 
overwhelmingly positive on the public blog 
in response to the specific interdisciplinary 
learning question. We have two thoughts about 
this conflicting feedback: 1) Students may have 
been reluctant to report negative attitudes in 
a public blog due to the fear of damaging the 


strong supportive relationships that they had built, 
regardless of discipline, during the course; 2) 
With the exception of the one blogging question 
about interdisciplinary work late in the semester, 
nothing in any of the assignments asked students 
to consider an interdisciplinary perspective. 
We speculate that students identified less with 
their discipline affiliation and more with their 
graduate student status, unified by graduate school 
expectations and their current work-life demands 
as students, employees, and family members. 
Despite these commonalities, we suspect that the 
students nonetheless, and perhaps unconsciously, 
brought their different disciplinary perspectives 
and educational backgrounds into the course which 
subsequently increased the opportunity for diversity 
that they clearly recognized and applauded in much 
of their feedback. Unfortunately, by not explicitly 
addressing the interdisciplinary aspects throughout 
the course, we missed the opportunity to provide a 
richer level of reflective, and perhaps more diverse, 
learning experiences. Our lesson here is to take 
full advantage of the interdisciplinary nature of the 
course by asking students to compare and contrast 
disciplinary perspectives in some if not all of the 
assignments and activities. Finally, given their 
general compatibility, our experiences lead us to 
the additional lesson that differences among the 
disciplines in this FLC can be leveraged without 
provoking anxiety or confusion among either 
instructors or students. 

Openly-Networked Structure and Platform 

We have three observations about the openly- 
networked platform experience. First, both 
instructors and students found the WordPress 
learning curve steeper than anticipated, despite 
assistance from the instructional design member, 
the availability of a multitude of WordPress 
tutorials, and their own expertise in Blackboard. 
The multiple-linked web site structure, created for 
increased exposure to connected learning tools, 
may have been yet another complication. Feelings 
of being “lost in social space” are not unique to 
our experiences. Students attempting to navigate 
a similar open source social networking platform 
have complained about disorientation, confusion, 
the lack of structure, and difficulty finding things 
(Dron & Anderson, 2009). A simpler web site 
structure, with instructor and student users situated 
in just one course web site rather than multiple web 
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sites, might have been preferable. 

Although it is difficult to know the extent to 
which the steepness of the learning curve was due 
to the instructors’ limited exposure to web site 
design or to something inherent in this particular 
WordPress platform, one of the most important 
lessons here would be that instructors should gain 
at least minimal experience with web site design 
prior to, rather than concurrent with, experimenting 
with the delivery of an openly-networked course, 
especially in a content area as challenging as 
research methods. 

Second, we were naive to think that we could 
blend our experiences and knowledge about 
teaching research methods, particularly the need 
for instructor-imposed structure and guidance, with 
the openly-networked premise that learners create 
and assess their own learning pathways using an 
ever-increasing amount of digital information (de 
Laat, Lally, Lipponen, & Simons, 2007; Goodyear, 
Jones, Asensio, Hodgson, & Steeples, 2005; 
McConnell, Lally, & Banks, 2004). Rather than 
embracing the array of learning opportunities 
afforded by a combination of an open web site and 
a closed learning management system, students 
were confused, frustrated, and overwhelmed by 
the various technological formats which, in turn, 
created more demand for instructor guidance—and 
reassurance—about meeting course expectations. 
For example, early in the semester, students 
had difficulty learning from the self-assessment 
quizzes; they preferred ready answers to questions 
rather than exploring the material on their own. 
The pattern of requests for the librarian’s help 
suggests students continued to have apprehensions 
about searching the web, which abated only after 
we generated customized podcasts and summary 
blogs. 

It could be argued that the number and type of 
required course assignments contradicted the self¬ 
learning premise of openly-networked learning 
experiences and probably increased the students’ 
anxiety with the course structure. Furthermore, 
responding to many course assignments is time 
and labor intensive for students, leaving little 
opportunity for them to explore their own interests 
or specific learning needs. Nonetheless, it is a 
reality that research methods is a required course in 
several graduate professional education programs 
and that there is perhaps little negotiation about the 


basic research knowledge that is needed for success 
in subsequent research courses. Most importantly, 
the mandate for professional graduate students 
to build the critical research capacity needed for 
effective and responsible practice dictates that 
students demonstrate a certain level of foundational 
research knowledge, which, at this level of study, 
might be less subject to interpretation and differing 
perspectives than other topics. Ultimately, the 
instructors concluded that their responsibility to 
structure the learning process and assess learning 
outcomes, plus the student requests for increased 
instructor involvement, trumped the autonomous 
learner-discovery approach associated with openly- 
networked learning. 

Similarly, our third observation relates to the 
assumed relationship between connectivist learning 
and openly-networked platforms. Although there 
are theoretical arguments, particularly within 
the realm of connectivism, that promote openly- 
networked platforms (Dabbagh, 2007; Kop, 2011; 
Mentor, 2007), our experiences suggest otherwise. 
There is no evidence that the majority of the 
learning that occurred in this course resulted from 
an openly-networked platform. Rather, students 
reported that the bulk of their learning, as well 
as emotional support, derived from their small 
group experiences as they worked together on 
their worksheets and research proposals within 
the closed learning environment in Blackboard. 
Furthermore, student explorations in web-based 
materials and resources and in digital technologies 
central to networked learning were minimal and 
limited to those required by course assignments. 
In this regard, our students’ experiences were 
similar to others in openly-networked platforms 
in that most students did not work beyond course 
requirements (Saadatmand & Kumpulainen, 
2014; Veletsianos & Navarrete, 2012) and became 
confused and overwhelmed by an abundance of 
learning activities (Kahnwald & Pscheida, 2015; 
Miller, 2014) and technology tools (Fini, 2009). 

As to why this might be so, we tend to agree 
with Kop, Fournier, and Mak (2011) who question 
whether adult learners have the level of skill 
and investment to be effective network learners. 
They assert that students successful in networked 
learning need to be autonomous learners who have 
advanced analytic and synthesis skills to distill 
the abundance of network information and who 
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have a high level of competency and interest in 
using an array of digital tools (p. 74). Although it 
is tempting to recommend that graduate students 
in professional programs attain a greater level 
of digital proficiency and interest to meet these 
standards, such recommendations may not be 
feasible given what we know about online students: 

Online students are older... and have 
several responsibilities in life; 68% are 
using online programs to balance work, 
family, and school responsibilities while 
73%> are employed full-time and 16%> part- 
time, seeking convenience and flexibility 
when furthering their education for career 
advancement. (Aslanian & Ciinefeiter, 

2012, p. 16). 

Miller (2014) refers to online courses as the 
“third shift” that students engage in after their paid 
work and their family work. There is little to suggest 
that such a group of students would have the time 
or the inclination to amass the skill necessary to 
succeed in a true openly-networked course. 

The lesson here is that required research methods 
courses in professional education programs may 
not be the most suitable ones to express the full 
benefits of open and connected learning platforms. 
While we tried to attend to the “open, emergent, 
chaotic nature of online interaction” characteristic 
of many open-networked courses, we tend to agree 
with Kop, Fournier, and Mak (2011) that such 
an approach might conflict with the organized 
structure of formal education, which involves 
prescriptive learning and standardized goals and 
curricula. We suspect that the nature of teaching 
and learning research methods, as we know it now, 
may be less consistent with the student-centered 
constructivist model and more aligned with an 
instructivist approach (Dron & Anderson, 2009). 

Blogging Component 

Despite widespread support for blogging 
among proponents, Sim and Hew (2010) found that 
rigorous studies demonstrating the educational 
value of blogs are scarce. Based on their review 
of 24 empirically-based articles, they concluded 
that “the question on whether the use of blogs can 
improve performance outcome such as learning or 
thinking is still unresolved” (p 157). Similarly, we 
found that public blogging offered little in terms 
of learning in this ISRM course. To reiterate, 


students attributed their major learning to the 
small-group work encapsulated within Blackboard 
and seemed burdened by the blogging component. 
As instructors, we were especially concerned about 
the inaccurate or misinterpreted research content 
shared in the blogs and the inability of students to 
recognize and/or correct these content errors in a 
public space. 

Furthermore, our students concurred with 
graduate students elsewhere who expressed 
concerns about the public nature of their blogs 
how others might perceive them online. They 
worried about the time they spent keeping up with 
other’s blogs, making comments, and trying to find 
supplementary material for their own blog posts 
(Veletsianos & Navarrete, 2012). Although we are 
tempted to offer but one take-away lesson—not to 
utilize a public blogging component in graduate 
research methods courses—we believe that 
such a blanket statement might be short-sighted, 
especially given the speed of technological and 
pedagogical changes we are witnessing. Rather, we 
recommend careful consideration of the fit between 
subject content, pedagogy, and the technological 
assumptions that drive new learning models 
(Mishra & Koehler, 2006). Most importantly, we 
remain open to the possibility that research methods 
courses may need to be completely restructured to 
better fit self-guided learning environments unlike 
anything that we know today, where the public 
blogging component serves as the primary, and 
perhaps sole, teaching and learning strategy. 

Information Competencies 

The librarian’s role grew more active and 
collaborative as her expertise in the literature 
review process and proposal development became 
increasingly apparent. The search exercise, 
videos, and handouts emphasized the systematic 
and iterative nature of the search process and 
demonstrated how research questions and 
hypotheses can change following a more extensive 
literature review. Most importantly, we suspect that 
the librarian’s consultations reduced student anxiety 
around the completion of the proposal worksheets 
in particular and in the research process in general. 
Consistent with the library and information science 
literature (Mounce, 2010; Rempel & Davidson, 
2008), the major lesson here is to better broadcast 
and take advantage of the effectiveness of faculty- 
librarian collaborations in helping students develop 
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research competencies. 

CONCLUSION 

Our exploration with an interdisciplinary 
openly-networked graduate level computer- 
mediated course provided mixed results for 
the educational experiences that we had hoped 
to achieve. In that sense, we suggest that the 
glass is half-full. We provide evidence to pursue 
interdisciplinary research methods courses, to 
examine the “constellation” of teaching activities 
during the collaborative planning process, and to 
showcase the interdisciplinary nature of the course. 
We illuminate the benefits of librarian-research 
faculty partnerships in building both information 
competencies and research capacity. The ability 
of the closed Blackboard system to provide 
sufficient connectivity for meaningful, interactive 
online learning is affirmed. On the other hand, we 
found the public blogging aspect and the openly- 
networked components, at least the way they were 
organized and presented within our course, to be 
problematic and not suitable for duplication in 
graduate research methods courses without more 
extensive examination. 
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